Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🎊 Feature: Get and Store Splits 🎊 #32

Merged
merged 76 commits into from
Aug 27, 2020
Merged

🎊 Feature: Get and Store Splits 🎊 #32

merged 76 commits into from
Aug 27, 2020

Conversation

suchak1
Copy link
Owner

@suchak1 suchak1 commented Aug 27, 2020

Get split data from IEX Cloud and Polygon.io and store in S3.

suchak1 and others added 22 commits August 6, 2020 22:11
* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows
* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes
@suchak1 suchak1 added the enhancement New feature or request label Aug 27, 2020
@suchak1 suchak1 linked an issue Aug 27, 2020 that may be closed by this pull request
@suchak1 suchak1 merged commit 469a6f2 into master Aug 27, 2020
@suchak1 suchak1 deleted the feature/get-splits branch October 15, 2020 23:36
suchak1 added a commit that referenced this pull request Jan 13, 2021
* Feature: init function (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* Add logo (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* Bug: Fix Auth Flow / Add 2FA (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* Feature: Get and Store Dividends (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* Feature: S3 Integration (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💽 Feature: Cache pip dependencies 💽 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* still need to write get_splits fx

* fix save_symbols test

* fixing tests

* tests fixed?

* iex get splits done

* splits functions done

* adding splits update to the pipeline

* writing tests

* higher order fx refactor and almost done w tests

* tests done?

* fix test_get_splits_path

* fix columns naming

* refactor

* try again
suchak1 added a commit that referenced this pull request Jan 13, 2021
… ♊ (#78)

* 🎬 chore: init function 🎬 (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* 🏹 chore: add logo 🏹 (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* 📱 fix(broker): auth flow / 2FA 📱 (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* 💰 feat(data): dividends 💰 (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* 💽 feat(data): s3 integration 💽 (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💾 chore(auto): cache pip dependencies 💾 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* 💎 feat(data): add polygon.io 💎 (#23)

* still need to write get_splits fx

* made iexcloud subdirs

* start polygon class

* fix test

* init function

* switch to polygon api

* auto dep check

* polygon dividend redundancy added

* tests

* update env var in workflows

* secrets

* fixes?

* fix test

* done

* Bump robin-stocks from 1.3.0 to 1.4.2 (#25)

* Feature: init function (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* Add logo (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* Bug: Fix Auth Flow / Add 2FA (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* Feature: Get and Store Dividends (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* Feature: S3 Integration (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💽 Feature: Cache pip dependencies 💽 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* 💎 Feature: Add Polygon.io (Double Redundancy) 💎 (#23)

* still need to write get_splits fx

* made iexcloud subdirs

* start polygon class

* fix test

* init function

* switch to polygon api

* auto dep check

* polygon dividend redundancy added

* tests

* update env var in workflows

* secrets

* fixes?

* fix test

* done

* Bump robin-stocks from 1.3.0 to 1.4.2

Bumps [robin-stocks](https://github.com/jmfernandes/robin_stocks) from 1.3.0 to 1.4.2.
- [Release notes](https://github.com/jmfernandes/robin_stocks/releases)
- [Commits](https://github.com/jmfernandes/robin_stocks/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: Krish Suchak <42231639+suchak1@users.noreply.github.com>
Co-authored-by: Krish Suchak <suchak.krish@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump python-dotenv from 0.13.0 to 0.14.0 (#26)

* Feature: init function (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* Add logo (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* Bug: Fix Auth Flow / Add 2FA (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* Feature: Get and Store Dividends (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* Feature: S3 Integration (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💽 Feature: Cache pip dependencies 💽 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* 💎 Feature: Add Polygon.io (Double Redundancy) 💎 (#23)

* still need to write get_splits fx

* made iexcloud subdirs

* start polygon class

* fix test

* init function

* switch to polygon api

* auto dep check

* polygon dividend redundancy added

* tests

* update env var in workflows

* secrets

* fixes?

* fix test

* done

* Bump python-dotenv from 0.13.0 to 0.14.0

Bumps [python-dotenv](https://github.com/theskumar/python-dotenv) from 0.13.0 to 0.14.0.
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/master/CHANGELOG.md)
- [Commits](theskumar/python-dotenv@v0.13.0...v0.14.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: Krish Suchak <42231639+suchak1@users.noreply.github.com>
Co-authored-by: Krish Suchak <suchak.krish@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump boto3 from 1.14.35 to 1.14.46 (#27)

* Feature: init function (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* Add logo (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* Bug: Fix Auth Flow / Add 2FA (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* Feature: Get and Store Dividends (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* Feature: S3 Integration (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💽 Feature: Cache pip dependencies 💽 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* 💎 Feature: Add Polygon.io (Double Redundancy) 💎 (#23)

* still need to write get_splits fx

* made iexcloud subdirs

* start polygon class

* fix test

* init function

* switch to polygon api

* auto dep check

* polygon dividend redundancy added

* tests

* update env var in workflows

* secrets

* fixes?

* fix test

* done

* Bump boto3 from 1.14.35 to 1.14.46

Bumps [boto3](https://github.com/boto/boto3) from 1.14.35 to 1.14.46.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.14.35...1.14.46)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: Krish Suchak <42231639+suchak1@users.noreply.github.com>
Co-authored-by: Krish Suchak <suchak.krish@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* 🤖 fix(auto): dependabot race condition 🤖 (#30)

* Feature: init function (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* Add logo (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* Bug: Fix Auth Flow / Add 2FA (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* Feature: Get and Store Dividends (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* Feature: S3 Integration (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💽 Feature: Cache pip dependencies 💽 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* still need to write get_splits fx

* fix save_symbols test

* fixing tests

* tests fixed?

* run nonce

* kwargs

* save_symbols fix

* run id

* again

* 🎊 feat(data): splits 🎊 (#32)

* Feature: init function (#7)

* new files

* create symbol / instrument lookup tables

* init or new?

* abstract a few functions out

* historical data

* update readme to reference roadmap

* update reqs

* newline for function

* Add logo (#8)

* add logo

* adding logo

* update gitignore

* update readme

* Create python-app.yml

* organization and linting

* pytest-check

* flatten tests

* flatten tests

* new tests

* test command update

* just run pytest

* init and copy token bash script

* env vars

* workflow

* secrets

* progress towards automation

* handled non ci case

* update token script almost done

* chmod +x

* make tokens dir

* split prebuild and postbuild scripts

* fix workflow

* script file path in workflow

* housecleaning

* get windows username on wsl

* new workflow

* rename workflows

* yml syntax

* schedule

* try again

* try again

* daily update

* update todo

* code coverage

* import sys

* lower coverage threshold

* test_load_portfolio

* auto versioning

* writing last tests

* tests done

* locked reqs

* wrong dotenv

* try again

* update function name

* fix interval def

* comment out csv creation

* yml descriptions

* 🐞 Bug: Commit New OAuth 🐞 (#12)

* debug workflow

* add email and password env vars

* debug echo

* more debugging

* single line

* see if token exists in project dir

* try quotes

* fixed?

* diff to cmp

* double login

* log cmp result

* use quotes

* fixed issue

* try again

* whyyy

* unsilence gpg

* encrypt in root dir

* pass ref

* force=true

* try different action

* try different action

* [AUTO] OAuth token refresh #patch

* remove comments

* force push option

* [AUTO] OAuth token refresh #patch

* Update update.yml

* Bug: Fix Auth Flow / Add 2FA (#16)

* change env var names

* new auth flow

* fix 2fa

* add 2fa env var to workflow

* readme and auto release

* Feature: Get and Store Dividends (#14)

* script updates

* stricter branch cov & csv utils

* csv tests

* data dir

* typo

* fix test

* fix test data object

* uncomment tests

* yfinance import

* add local token update script

* symbols.csv

* script changes

* flake

* update cron schedule

* fix hist test?

* [AUTO] OAuth token refresh #patch

* comment out print statement

* only save hist purposefully

* only auto commit token

* yaml

* token from master

* readme

* readme again

* create datareader and datawriter classes

* update login script

* broker superclass tangent

* reorganization

* flake8

* dividend function done

* try auto symbol

* try again

* [AUTO] Symbol update #patch

* #minor commit

* auto dividend

* fixed robinhood dot to yahoo dash bug

* add multiprocessing

* fixed filtering for dividend and splits tables

* fixed csv update bug + more robust dividend saving

* div update

* div update iex

* constants

* debug

* working on tests

* working broker tests

* remove yfinance dep

* broker tests done

* fix naming issue

* import

* working on marketdata tests

* finished marketdata tests

* fileops tests almost done

* done

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* [AUTO] Symbol update #patch

* Feature: S3 Integration (#19)

* fix release?

* file walk

* storage file

* s3 dir uploader working as expected

* dividend s3 upload script complete

* dividend workflows and weird dividend bug

* floating point precision fix

* remove data dir

* move pathfinder to fileops

* s3 integration

* add functions to store class and finish most store tests

* finish store tests

* tests done?

* done?

* linting

* aws configure

* make path before saving

* circular import

* make path

* remove pathlib

* fix rename

* more rename fixes

* does this work

* done

* fixed workflows

* hotfix

* 💽 Feature: Cache pip dependencies 💽 (#20)

* refactor workflows

* pip cache

* add workflow badges to readme

* spacing

* more workflow and readme changes

* still need to write get_splits fx

* fix save_symbols test

* fixing tests

* tests fixed?

* iex get splits done

* splits functions done

* adding splits update to the pipeline

* writing tests

* higher order fx refactor and almost done w tests

* tests done?

* fix test_get_splits_path

* fix columns naming

* refactor

* try again

* Bump robin-stocks from 1.4.2 to 1.5.0 (#31)

Bumps [robin-stocks](https://github.com/jmfernandes/robin_stocks) from 1.4.2 to 1.5.0.
- [Release notes](https://github.com/jmfernandes/robin_stocks/releases)
- [Commits](https://github.com/jmfernandes/robin_stocks/commits)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump anothrNick/github-tag-action from 1.22.0 to 1.26.0 (#33)

Bumps [anothrNick/github-tag-action](https://github.com/anothrNick/github-tag-action) from 1.22.0 to 1.26.0.
- [Release notes](https://github.com/anothrNick/github-tag-action/releases)
- [Commits](anothrNick/github-tag-action@1.22.0...9aaabdb)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump boto3 from 1.14.46 to 1.15.5 (#38)

Bumps [boto3](https://github.com/boto/boto3) from 1.14.46 to 1.15.5.
- [Release notes](https://github.com/boto/boto3/releases)
- [Changelog](https://github.com/boto/boto3/blob/develop/CHANGELOG.rst)
- [Commits](boto/boto3@1.14.46...1.15.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump pandas from 1.1.0 to 1.1.2 (#39)

Bumps [pandas](https://github.com/pandas-dev/pandas) from 1.1.0 to 1.1.2.
- [Release notes](https://github.com/pandas-dev/pandas/releases)
- [Changelog](https://github.com/pandas-dev/pandas/blob/master/RELEASE.md)
- [Commits](pandas-dev/pandas@v1.1.0...v1.1.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump polygon-api-client from 0.1.6 to 0.1.8 (#36)

* Bump polygon-api-client from 0.1.6 to 0.1.8

Bumps [polygon-api-client](https://github.com/polygon-io/client-python) from 0.1.6 to 0.1.8.
- [Release notes](https://github.com/polygon-io/client-python/releases)
- [Commits](polygon-io/client-python@v0.1.6...v0.1.8)

Signed-off-by: dependabot[bot] <support@github.com>

* Update requirements.txt

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Krish Suchak <42231639+suchak1@users.noreply.github.com>

* 🐦 feat(data): social sentiment 🐦 (#40)

* resolved

* get_ohlc iexcloud

* dep updates

* get_ohlc polygon

* refactor ohlc

* stocktwits

* comment

* progress

* scripts

* finishing up implementation

* workflows

* fixed

* update_sentiment mods

* done

* pipeline env vars

* changes

* fixed

* pls

* 🐛 fix(auto): sentiment schedule 🐛 (#42)

* resolved

* fixed

* workflow name

* fix readme

* 💾 fix(data): save functions 💾 (#45)

* resolved

* deps

* fix

* temp fix

* try again

* 🏖️  fix(test): temp IEX sandbox fix 🏖️ (#46)

* resolved

* deps

* fix

* temp fix

* try again

* notlikethis

* 📈 feat(data): ohlc 📈 (#47)

* resolved

* get_ohlc iexcloud

* dep updates

* get_ohlc polygon

* refactor ohlc

* stocktwits

* comment

* progress

* scripts

* finishing up implementation

* workflows

* semantics

* undo sandbox changes

* upgrade rh

* delete extra files

* add get_ohlc tests

* add standardize_ohlc test

* add save_ohlc test

* add get_ohlc for iex cloud

* add get_ohlc for polygon

* fix get_ohlc for polygon

* add get_ohlc_path test

* add save_ohlc to marketdata

* add get_ohlc to marketdata

* comment out unused code

* lower test coverage threshold from 95 to 90

* 📱 feat(data): crypto social sentiment 📱 (#48)

* resolved

* add mergers to planned data collection in readme

* add crypto tickers to sentiment update

* readme

* ✋ fix(test): stop uploading test data during repo upload ✋ (#49)

* resolved

* add mergers to planned data collection in readme

* done

* 🧵 Ffeat(data): multiprocessing update 🧵 (#50)

* resolved

* add mergers to planned data collection in readme

* add get_intraday_path

* comments

* add institutional sentiment to readme data checklista

* to compile

* limit polygon save timeframe to prevent retroactive weirdness

* add multiprocessing to speed up data updates

* delete local data files after each file update to preserve disk space

* linting

* if

* ❌ chore: rename repo ❌ (#51)

* resolved

* add mergers to planned data collection in readme

* add get_intraday_path

* comments

* add institutional sentiment to readme data checklista

* to compile

* limit polygon save timeframe to prevent retroactive weirdness

* add multiprocessing to speed up data updates

* delete local data files after each file update to preserve disk space

* constants tests done

* conflict

* conflict

* polygon test_intraday

* iex test_intraday

* test_save_intraday + repo rename

* readme

* fix tests

* 💱 feat(data): crypto ohlc + 3 retries 💱 (#52)

* making progress

* try again function

* iex div and splits retry

* poly div and splits retry

* iex ohlc retry

* tests updated

* fix try again

* bugs

* fix tests

* fix retries for tests

* add polygon key

* timezone fix

* 🌲 fix: env var issue 🌲 (#53)

* 📦 chore(auto): publish package on pypi 📦 (#58)

* done?

* upgrade packages

* 🥡 fix(auto): pypi package upload 🥡 (#60)

* done?

* upgrade packages

* fix?

* python

* ⚖️ feat(data): volume weighted average ⚖️ (#61)

* resolved

* add mergers to planned data collection in readme

* add get_intraday_path

* comments

* add institutional sentiment to readme data checklista

* to compile

* limit polygon save timeframe to prevent retroactive weirdness

* add multiprocessing to speed up data updates

* delete local data files after each file update to preserve disk space

* constants tests done

* conflict

* conflict

* polygon test_intraday

* iex test_intraday

* test_save_intraday + repo rename

* readme

* git cleanup local branches

* package name change

* remove package upload in dev pipeline

* fix bugs and add volume weighted avg price to ohlc

* tests

* lint

* lint again

* pandas warning

* fix date polygon

* 🔢 feat(data): number of trades 🔢 (#66)

* update deps

* change symbol?

* remove error log

* debug

* update hist scripts to get s3 truth

* bug fix

* 📅 feat(data): intraday 📅 (#71)

* resolved

* add mergers to planned data collection in readme

* add get_intraday_path

* comments

* add institutional sentiment to readme data checklista

* to compile

* limit polygon save timeframe to prevent retroactive weirdness

* add multiprocessing to speed up data updates

* delete local data files after each file update to preserve disk space

* constants tests done

* conflict

* conflict

* polygon test_intraday

* iex test_intraday

* test_save_intraday + repo rename

* readme

* git cleanup local branches

* package name change

* remove package upload in dev pipeline

* fix bugs and add volume weighted avg price to ohlc

* start generator

* polygon intraday done?

* more progrress

* fix polygon, still need to do iex

* finish iex changes

* add intraday step to dev pipeline and fix tests

* fixed?

* last fix?

* get date from first row of intra to save properly

* more inclusive time range and lower threshold

* 🕰️ fix(data): stock datetime formatting 🕰️ (#72)

* fix formatting

* try again

* lint

* 💱 fix(data): crypto datetime 💱 (#76)

* stop using eod ohlc path for standardize intraday

* missing comma

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store Splits and Investigate Non Cash Dividends
1 participant